1. Technical Field
The present invention relates in general to a system and method for adjusting operating system read ahead values based on memory usage. More particularly, the present invention relates to a system and method for monitoring virtual memory conditions and adjusting read ahead values associated with reading sequentially accessed files.
2. Description of the Related Art
Virtual memory is important in many modern, complex operating systems. Virtual memory is an imaginary memory area supported by some operating systems (for example, IBM's AIX™ operating system) in conjunction with the hardware. Virtual memory provides an alternate set of memory addresses. Programs use these virtual addresses rather than real addresses to store instructions and data. When the program is actually executed, the virtual addresses are converted into real memory addresses.
The purpose of virtual memory is to enlarge the address space, the set of addresses a program can utilize. For example, virtual memory might contain twice as many addresses as main memory. A program using all of virtual memory, therefore, would not be able to fit in main memory all at once. Nevertheless, the computer could execute such a program by copying into main memory those portions of the program needed at any given point during execution.
To facilitate copying virtual memory into real memory, the operating system divides virtual memory into pages, each of which contains a fixed number of addresses. Each page is stored on a disk until it is needed. When the page is needed, the operating system copies it from disk to main memory, translating the virtual addresses into real addresses.
In AIX™, virtual memory segments are partitioned into 4K (4096) byte units called pages and real memory is divided into 4K-byte page frames. The VMM manages the allocation of page frames as well as resolving references to virtual-memory pages that are not currently in RAM (stored in paging space) or do not yet exist. To accomplish these tasks, the VMM maintains a “free list” of available page frames and uses a page-replacement algorithm to determine which virtual-memory pages that are currently in RAM will have their page frames reassigned to the free list (i.e., swapped out). The page-replacement algorithm used by AIX's VMM takes into account the pages that are “persistent” verses those that are “working segments.” As the name implies, persistent memory segments have permanent storage locations on disk. Data files or executable programs are typically mapped to persistent segments. On the other hand, working segments are transitory and exist only during their use by a program. Working segments have no permanent disk storage location. When working segments are paged out, they are written to disk paging space. When a program exits, all of the program's working pages are immediately placed back on the free list. Because working pages must be written back to disk before re-using its page frames, it is usually preferable to swap out persistent memory segments before swapping working memory segments.
Modern operating systems, such as IBM's AIX™ operating system, often use a Virtual Memory Manager (VMM) to manage the virtual memory in order to service memory requests from the operating system as well as memory requests received from applications. Many VMMs try to anticipate when a program is sequentially reading a file from disk in order to pre-fetch pages so that subsequent pages will already be loaded into memory before being requested by the program. This anticipation performed by the VMM is often referred to as “Sequential-Access Read Ahead.”
In AIX™, the VMM tries to anticipate the future need for pages of a sequential file by detecting the pattern in which a program is accessing the file. When the program access two successive pages of a file, the VMM assumes that the program will continue to access the file sequentially. Consequently, the VMM schedules additional sequential reads of the file so that the file data is available to the program sooner than if the VMM waited to initiate the file I/O until the program requested the next page from the file.
In AIX™, Sequential-Access Read Ahead can be turned on/off as well as tuned using two VMM thresholds. First, minpgahead is set to the number of pages that are read ahead when the VMM first detects access of a sequential file. The second tuning threshold, maxpgahead, is set to the maximum number of pages the VMM will read ahead in a sequential file. When a sequential file is first detected, minpgahead pages are read. When subsequent requests are made for additional pages of the sequentially accessed file, the number of pages that are pre-fetched is increased until maxpgahead pages are pre-fetched.
FIG. 1 is a diagram showing a prior art implementation of Sequential-Access Read Ahead. VMM Sequential-Access Read Ahead processing commences at 100 whereupon, at step 120, a first access of file 110 causes the first page (page 0) of file 100 to be read. At this point the VMM makes no assumption about random or sequential file access. At step 130, the program accesses the first byte of the next page (page 1), with no intervening accesses to other pages of the file. At this point, VMM concludes that the program is accessing sequentially. It schedules a number of extra pages (e.g., two extra pages) corresponding to the current minpgahead value. In this example, two additional pages (pages 2 and 3) are read. Thus, in this example a total of 3 pages are read as a result of the program's second read request.
At step 140, the program accesses the first byte of the first page that has been read ahead (page 2), the VMM doubles the page-ahead value to 4 and schedules pages 4 through 7 to be read from file 110.
At step 150, the program accesses the first byte of the first page that has been read ahead (page 4), the VMM again doubles the page-ahead value to 8 and schedules pages 8 through 15 to be read. This doubling continues until the amount of data being read reaches maxpgahead or until the end of the file is reached.
At step 160, maxpgahead has been reached and the VMM continues reading maxpgahead pages when the program accesses the first byte of the previous group of read-ahead pages until the file ends.
As can be seen by the diagram shown in FIG. 1, having a high maxpgahead value improves efficiency and speed of programs performing large amounts of sequential-access reads. A challenge, however, of pre-fetching large numbers of sequentially-accessed pages is that memory can sometimes become constrained. When memory becomes constrained, the VMM determines which virtual-memory pages that are currently in RAM will have their page frames reassigned to the free list. In the case of persistent memory segments, identified pages can be reassigned quickly. However, if working memory segments need to be reassigned, the working memory segment data must first be written to disk paging space.
What is needed, therefore, is a system and method for identifying memory constraint conditions and dynamically adjusting the VMM's Sequential-Access Read Ahead threshold values accordingly. Furthermore, what is needed is a system and method that automatically turns off the VMM's Sequential-Access Read Ahead when memory is highly constrained and turns the Sequential-Access Read Ahead in a manner that reduces the number of times the Sequential-Access Read Ahead is toggled between on and off states.